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ABSTRACT 


A  method  of  image  segmentation  has  been  developed  based 
on  creating  links  between  pixels  in  successive  layers  of  a 
"pyramid"  of  reduced-resolution  versions  of  the  image.  In 
the  original  implementation  of  this  method,  the  links  were 
based  on  comparing  the  values  of  a  single  feature,  (average) 
gray  level,  for  each  pixel.  In  this  note,  the  method  is 
extended  to  links  based  on  multiple  features,  such  as  color 


components  or  neighborhood  properties. 


The  support  of  the  Defense  Advanced  Research  Projects  Agency 
and  the  U.S.  Army  Night  Vision  Laboratory  under  Contract 
DAAG-53-76C-01p38  (DARPA  Order  3206)  is  gratefully  acknowledged, 
as  is  the  help  of  Sherry  Palmer  in  preparing  this  report. 


A  method  of  image  segmentation  described  in  [1]  is 
based  on  creating  links  between  pixels  in  successive  layers 
of  a  "pyramid"  of  reduced-resolution  versions  of  the  image. 
The  method  makes  use  of  an  "overlapped"  pyramid  in  which 
each  level  is  initially  obtained  by  averaging  4-by-4  blocks 
of  pixels  on  the  level  below ,  where  these  blocks  overlap  by 
50%  in  the  x  and  y  directions;  this  implies  that  each  pixel 
has  four  "fathers"  on  the  level  above  it.  To  segment  the 
image,  we  link  each  pixel  (at  each  level  of  the  pyramid)  to 
that  one  of  its  four  fathers  whose  gray  level  is  most  similar 
to  its  own.  We  now  recompute  the  value  of  each  pixel  (on  the 
levels  above  the  image  itself)  by  averaging  only  those  pixels 
on  the  level  below  that  are  linked  to  it.  Based  on  these 
new  values,  we  change  the  links  as  necessary;  recompute  the 
values  based  on  these  new  links;  and  so  on,  repeating  the 
process  until  there  is  no  further  change  (typically  after  a 
few  iterations) .  The  resulting  final  links  define  subtrees 
of  the  pyramid,  having  pixels  as  leaves  and  having  roots  at 
the  uppermost  level,  which  we  usually  take  to  be  the  2 -by- 2 
level.  Each  subtree  thus  defines  a  subset  of  the  pixels  (its 
leaves) ,  so  that  we  have  segmented  the  image  into  (at  most) 
four  subsets. 

The  linking  process  need  not  be  used  on  average  gray 
level,;  we  can  use  any  property  that  can  be  computed  for  th 
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common  gray  level  (four  widely  spaced  levels  were  used) . 

The  red  segmentation  does  not  distinguish  bushes  from 

shadowed  brick;  the  green  segmentation  breaks  the  bushes 

up  into  a  grass-like  and  a  shadow-like  part;  and  the 

blue  segmentation,  besides  not  distinguishing  bushes  from 

shadow,  breaks  the  sunlit  brick  into  arbitrary  subclasses. 

Figure  lc  shows  results  when  we  use  two  colors  at  a  time: 

green  and  blue  in  the  left  column,  blue  and  red  in  the 

center  column,  and  red  and  green  in  the  right  column.  Here, 

the  segmentations  confuse  parts  of  the  bush  class  with  the 

shadow  class ,  but  the  blue/red  segmentation  gives  a  fairly 

good  partitioning  of  the  image  into  the  four  types  of  regions 

Figure  2a  shows  a  local  "busyness”  measure  computed 

for  each  pixel  of  the  red,  green,  and  blue  images.  [For  the 

a  b  c 

3 -by- 3  neighborhood  d  e  f,  this  measure  is  defined  by 
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where  the  min  is  used  to  decrease  the  response  to  step  edges, 
while  retaining  a  high  response  in  isotropic  "busy”  regions.] 
Figure  2b  shows  segmentation  results  based  on  busyness  alone; 
the  green  result,  in  particular,  gives  fair  discrimination 
between  the  bushes  and  the  other  regions,  but  the  other 

n^egions  themselves  -  as  should  be  expected  -  are  not  distin- 

l  \ 

gulshed.  Figure  2c  shows  results  when  we  use  two  features, 
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|jf*yv  level  (in  the  given  band)  and  busyness;  none  of  these 


levels  of  the  pyramid  by  averaging.  If  desired,  we  can 
begin  the  process  by  dividing  the  image  into  blocks,  com¬ 
puting  a  property  (e.g.,  a  textural  property)  for  each 
block,  and  building  a  pyramid  starting  from  the  resulting 
array  of  property  values;  linking  in  this  pyramid  yields 
a  (blocky)  segmentation  of  the  image  into  (at  most  four) 
textured  regions  [2].  Other  modifications  of  the  basic 
pyramid  linking  process  have  been  investigated,  including 
the  use  of  weighted  rather  than  forced-choice  links  [3],  as 
well  as  variations  of  the  linking  criterion  that  take 
positional  information  into  account  [ 4 ] .  Methods  of  com¬ 
bining  this  linking  scheme  with  image  segmentation  by 
recursive  splitting  into  homogeneous  regions  have  also  been 
investigated  [ 5 ] . 

The  linking  process  need  not  be  based  on  a  single  property; 
we  can  compute  a  property  vector  for  each  pixel  (or  block) 
and  extend  it  to  the  higher  levels  of  the  pyramid  by  com¬ 
ponentwise  averaging.  This  note  illustrates  this  generalization 
with  some  simple  examples  of  pyramid  linking  based  on  pairs 
of  properties. 

Figure  la  shows  the  red,  green,  and  blue  bands  of  a  color 
image  of  part  of  a  house,  showing  sunlit  and  shadowed  brick, 
bushes,  and  grass.  Figure  lb  shows  the  results  of  the  pyramid 
linking  process  applied  to  each  band  separately;  the  pixels 
belonging  to  each  of  the  four  subtrees  are  displayed  with  a 


yields  the  four  types  of  regions,  but  the  results  in  the 
green  band  yield  the  brick  (both  sunlit  and  shadowed)  as 
essentially  a  single  region,  while  dividing  the  bushes 
into  two  classes.  All  results  shown  are  for  ten  itera¬ 
tions  . 

working  with  two  features  does  not  always  yield  signi¬ 
ficant  improvement  over  working  with  single  features. 

Table  1  shows  classification  results  for  two  geological 
terrain  types.  Lower  Pennsylvanian  Shale  and  Pennsylvanian 
Sandstone  and  Shale,  using  three  texture  features,  singly 
and  pairwise.  Each  feature  was  the  moment  of  inertia  of  a 
gray  level  co-occurrence  matrix  about  its  main  diagonal 
(Haralick's  "CON"  feature),  where  the  matrices  were 
defined  for  the  displacements  shown  in  the  second  column 
of  the  table.  In  earlier  texture  classification  studies 
using  the  same  terrain  types  [6] ,  these  features  individually 
had  error  rates  of  close  to  25%,  whereas  pairwise  they  gave 
error  rates  of  under  10%.  In  the  present  experiment, 
however,  there  is  little  or  no  improvement  when  pairs  are 
used,  probably  because  the  error  rates  for  single  features 
are  so  low. 

In  conclusion,  these  examples  illustrate  how  the  basic 
pyramid  linking  method  may  sometimes  yield  a  richer  class 
of  segmentations  by  using  more  than  one  property  to 


characterize  the  pixels.  Pyramid  linking  appears  to 
deserve  further  investigation  as  a  general  approach  to 
image  segmentation. 
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Feature  No. 

Displacement 

Error  rate(%) 

Feature  pair 

Error  rate(%) 

1 

(1/0) 

4.3 

2&3 

3.3 

2 

(0,1) 

6.4 

3&1 

5.2 

3 

(0,2) 

14.2 

1&2 

6.2 

Table  1.  Classification  results  for  geological 
terrain  types 


Figure  1.  a)  House  image:  red,  green  and  blue  components 

b)  Results  of  pyramid  linking  applied  to  each 
band  separately 

c)  Results  using  pairs  of  bands:  green/blue, 
blue/red,  red/green 


Figure  2.  a)  "Busyness"  values  in  the  three  bands 

b)  Results  of  pyramid  linking  applied  to  these  values 

c)  Results  using  (intensity,  busyness)  feature  pair 
in  each  band 


UNCLASSIFIED _ 

security  euAWtriCATiow  or  this  rape  fl**—  o«f 

REPORT  DOCUMENTATION  PAGE 

I.  REPORT  NUMBER  |2.  OOVT  ACCESSION  NO. 

_ AO-  hvaM^tS 


4.  TITLE  (mt4  SvMtl») 

MULTIBAND  PYRAMID  LINKING 


T.  AUTHOR?*) 

Tsai-Hong  Hong 
Azriel  Rosenfeld 

PERPORMINa  ORGANIZATION  NAME  AND  ADDRESS 

Computer  Science  Center 

University  of  Maryland 

College  Park.  MD  20742 _ 

II.  CONTROLLING  OFFICE  NAME  AND  ADDRESS 

U.S.  Army  Night  Vision  Laboratory 
Fort  Belvoir ,  VA  22060 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 
RECIPIENT'S  CATALOO  NUMBER 


S.  TYRE  OP  REPORT  4  PERIOO  COVEREO 


Technical 

PERFORMING  oro.  report  number 

TR-1025 

B.  CONTRACT  OR  GRANT  NUMSERfa) 

DAAG-53-76C-0138 


PROORAM  CLEMENT.  PROJECT.  TASK 
AREA  4  WORK  UNIT  NUMBERS 


II.  REPORT  DATE 

March  1981 


’TMNiToElNa-ASlNC7irGiciAOOiiIii?7r27tarwiriiwrC«itt»l7jiiOinM>  I  It.  SECURITY  CL  ASS.  (ol  IM*  npeit) 


PlCATlON/DOWNORAOtNO 


OISTRI NOTION  STATEMENT  (•!  thlt  l 


Approved  for  public  release;  distribution  unlimited. 


I  IT.  DISTRIBUTION  STATEMENT  (ml  ARM  «NrMM  IM  M.  If  SMtawl  4m  RrmH; 


1 14.  SUPPLEMENTARY  NOTES 


IS.  KEY  WORDS  (GtmUmm  m  iwwm  KR  II  mmup  M  MmMS  4p  Mm4  mp4w) 

Image  processing  Texture 

Pattern  recognition 
Segmentation 
Pyramids 

l  CQlQJC  . . .  . . . . . . . 

B.  ABSTRACT  (SSiiSmm  m  wwiN  NR  u  im»«»ty  uM  irmi»  4y  41—4  mmtmr) 

^A  method  of  image  s  “gmentati  n  has  been  developed  based  on  creating 
links  between  pixel  sr  assive  layers  of  a  /^pyramid^  of  reduced- 
resolution  versions  r*  the  image.  In  the  original  implementation  of 
this  method,  the  links  were  based  on  comparing  the  values  of  a  single 
feature,  (average)  gray  level,  for  each  pixel.  In  this  note,  the 
method  is  extended  to  links  based  on  multiple  features,  such  as  color 
components  or  neighborhood  properties.. 


I  J  ANTS  W73  EDITION  op  I  NOV  «s  IS  OBSOLETE 


classified 


